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(54) ROBOT MONITORING DEVICE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide a monitoring device for monitoring the 
operation of a robot by a system different from robot control concerned. 
SOLUTION: The watching ranges 14 for watching the invasion of a robot are preset 
near robots 12, the watching ranges 14 are picked up for each specified time by using 
image pickup means 16, and whether a moving object exists in a part corresponding to 
the watching range 14 in an image picked up by the image pickup means 16 is judged 
by an analyzing means. The analyzing means judges the presence or absence of a 
moving object by detecting the movement of a part corresponding to the watching 
range 14 in the image. When it is judged that a moving object exists in the watching 
range 14, supply of power source to the robot 12 is stopped, or warning sound and/or 
warning light is emitted by providing a warning means for emitting warning sound 
and/or warning light on a monitoring device, and therefore, generation of abnormalities 
is preliminarily prevented. 
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CLAIMS 



[Claim(s)] 

[Claim 1] Robot supervisory equipment characterized by having an analysis means 
(18) to distinguish whether the migration body exists in the part equivalent to the 
watch field in the image photoed with an image pick-up means (16) to photo the watch 
field (14) which looks out for a robot's invasion set up near the robot (12) for every 
predetermined time, and this image pick-up means. 

[Claim 2] the difference of the image data of the image part in which an analysis 
means (18) is equivalent to the watch field between the minute time amount delta t 
(14) — the robot supervisory equipment according to claim 1 which detects lost 
motion and distinguishes the existence of the migration body in a watch field. 
[Claim 3] the difference of the image data of the image part in which an analysis 
means (18) is equivalent to the watch field between the minute time amount delta t 
(14) — the robot supervisory equipment according to claim 2 which detects a motion 



of a migration body and distinguishes the existence of a migration body by generating 
a lost-motion vector. 

[Claim 4] An analysis means (18) is robot supervisory equipment according to claim 1 
to 3 which transmits the instruction made to suspend supply of a power source to a 
robot (12) when the migration body existed all over the watch field (14) and it 
distinguishes. 

[Claim 5] An analysis means (18) is robot supervisory equipment according to claim 1 
to 4 which transmits the instruction which emits an alarm tone and/or alarm light to 
said alarm means (44) when it connected with the alarm means (44) which emits an 
alarm tone and/or alarm light, and the migration body existed all over the watch field 
(14) and an analysis means (18) distinguishes. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the equipment which supervises 

robots, such as an industrial robot. 

[0002] 

[Description of the Prior Art] Generally they equipped the interior with supervisors, 
such as a watchdog timer which controls actuation, and when the fault of a program of 
operation, disturbance, etc. separated and operate from actuation of a request of a 
robot by the cause, robots, such as an industrial robot, judged with "the overrun", and 
actuation of a robot is stopped or they have emitted the alarm. 
[0003] 

[Problem(s) to be Solved by the Invention] However, when the supervisor and the 
program which controls actuation of a robot were controlled by the same CPU in 
many cases and an error arose in CPU, even if the robot hung up, a supervisor might 
not fully operate. Moreover, since a supervisor was what supervises the robot's itself 
actuation, even if migration bodies other than the robots concerned, such as a man 
and other robots, approached the robot, it has not recognized it and was not able to 
prevent the collision with the migration body which approaches, and the robot 
concerned. 

[0004] The purpose of this invention is offering the supervisory equipment which can 
supervise actuation of a robot by different system, can emit an alarm when a robot's 
deviates from a predetermined working area and operates, supply of a power source 
to a robot was stopped, actuation of a robot is stopped or other migration bodies' 
approach a robot, and can prevent a collision of a migration body and a robot. 



[0005] 

[Means for Solving the Problem] In order to solve the above-mentioned technical 
problem, the robot supervisory equipment (10) of this invention The watch field (14) 
which looks out for invasion of the robot concerned near the robot (12) is set up 
beforehand. It distinguishes whether migration bodies including the robot (12) 
concerned exist in the part equivalent to the watch field (14) in the image which 
photoed this watch field (14) for every predetermined time using the image pick-up 
means (16), and was photoed with the image pick-up means (16) with an analysis 
means (18). An analysis means (18) distinguishes the existence of a migration body by 
detecting a motion of the part equivalent to the watch field in a screen (14). For 
example, a migration body can be recognized by generating the motion vector of the 
part equivalent to the watch field in a screen (14), and distinguishing the existence of 
the motion vector whose magnitude in a watch field is not zero. The following control 
is performed, when the migration body existed in the watch field (14) and an analysis 
means (18) distinguishes. For example, when the invasion direction of a migration body 
is a direction which goes to a watch field from a robot side, the migration body which 
trespassed upon the watch field (14) is the robot (12) concerned, and since it is 
thought that the robot (12) has hung up, supply of a power source to a robot (12) is 
stopped. Moreover, when the invasion direction of a migration body is a direction 
which faces to a robot, since migration bodies are a man and other robots, they 
prevent generating of abnormalities by forming the alarm means (44) which emits an 
alarm tone and/or alarm light in supervisory equipment (10), and emitting an alarm 
tone and/or alarm light. 
[0006] 

[Function and Effect] The robot supervisory equipment (10) of this invention monitors 
continuously the watch field (14) beforehand set up near the robot (12), detects a 
motion of the image in a watch field (14), and generates a motion vector. It can ask for 
an objective motion and an objective motion vector by calculating the difference of 
image data, such as a luminance signal in the continuous 2 inter-frame watch field for 
example, in minute time amount deltat. When a motion of a body is detected all over a 
watch field (14) (i.e., when the image which has the motion vector whose magnitude is 
not zero is detected), it considers that a robot (12) and other migration bodies have 
trespassed upon the watch field (14), and supply of a power source to a robot is 
suspended, or an alarm etc. is emitted. 

[0007] According to this invention, even when a robot hangs up and fault arises in the 
supervisor inside a robot, since the robot is supervising the robot by the separate 
system, he can prevent the overrun which became uncontrollable by the robot itself. 
Moreover, it can supervise not only about only a robot's monitor but about invasion of 
bodies, such as a man to the inside of a watch field, and other robots, and a collision 
etc. can be beforehand prevented by emitting an alarm etc. 



[0008] 

[Embodiment of the Invention] The robot supervisory equipment (10) of this invention 
is equipped with an image pick-up means (16) to photo the watch field (14) set up near 
the robot (12), and an analysis means (18) to analyze the image photoed with the 
image pick-up means (16) as shown in drawing 1 and drawing 2 . 

[0009] Image image pick-up equipments, such as a video camera and a digital still 
camera, can be used for an image pick-up means (16), and it is arranged so that it can 
supervise near the industrial robot (12) arranged along Rhine, such as works which 
perform assembly of a product on a band conveyor (19), from the upper part. Drawing 
3 A thru/or D express the image photoed by the image pick-up means (16). The field 
which a robot (12) can operate, respectively is set to the perimeter of an industrial 
robot (12), and the watch field (14) which looks out for a robot's invasion is set to the 
perimeter of the field which can be operated. In this example, a robot's (12)'s watch 
field (14) is set up as a back field and its headroom from a robot's side. 
[0010] An image pick-up means (16) photos the image (refer to drawing 3 A thru/or D) 
which includes a watch field in every minute time amount deltat, and transmits a 
picture signal to an analysis means (18). In addition, the number of pixels of an image 
pick-up means (16) is made into sxt (however, s and t both positive integer). As for 
minute time amount deltat, it is desirable to set it as the 1 field, the scan time of one 
frame, or its integral multiple, for example, it can make it 1 / 60 seconds. 
[0011] The image photoed by the image pick-up means (16) is inputted into an 
analysis means (18). The AD translation of the picture signal inputted into the analysis 
means (18) is carried out by the A-D converter (20). Next, the luminance signal of only 
the pixel corresponding to a watch field (14) is taken out from the luminance signal Y 
in the picture signal by which the AD translation was carried out. As an approach of 
taking out only the luminance signal of a watch field part, it has the storage section of 
the sxt individual corresponding to each pixel of an image for for example, the 
luminance signal Y, and "0" can mention the approach of integrating by the data and 
the AND gate (22) of the field distinction memory (24) written in beforehand to "1" 
and the other storage section at the storage section corresponding to the watch field 
(14) set up into the image. By integrating with the stored data with which field 
distinction memory (24) corresponds by the AND gate (22), luminance signals other 
than a watch field are outputted as zero. The luminance signal which passed the AND 
gate (22) is memorized by brightness memory (25). Brightness memory (25) has the 
storage section of the sxt individual corresponding to each pixel of an image, and 
memorizes the luminance signal inputted in order. 

[0012] Next, from an image pick-up means (16), it is transmitted to an analysis means 
(18) and the AD translation of luminance-signal Y' of the image after minute time 
amount deltat progress is carried out similarly. Inputted luminance-signal Y' passes an 
AND gate (22), and luminance signals other than a watch field serve as zero similarly, 



and it is outputted. 

[0013] Next, the difference of the luminance signal memorized by brightness memory 
and the luminance signal after time amount deltat is calculated in a difference circuit 
(26), and change of brightness is detected in the image in a watch field (14). Difference 
of the brightness calculated in the difference circuit (26) is absolute-value-ized in an 
absolute value-ized circuit (27), and when magnitude is not zero and "1" and 
magnitude are zero, it is distinguished from "0" in a binarization circuit (28). The pixel 
data in the watch field by which binarization was carried out are transmitted to a 
motion distinction circuit (30), and the existence of the existence of a migration body 
is distinguished. 

[0014] For example, as shown in B from drawing 3 A, when there is no change of an 
image all over a watch field (18) (i.e., when a migration body does not exist in a watch 
field) (the motion may arise in parts other than a watch field as shown in drawing), all 
are zero, and the difference of the luminance signal in a watch field serves as zero 
altogether, as the result which was absolute-value-ized and was carried out 
binarization is also shown in drawing 3 A-1. This result is transmitted to a motion 
distinction circuit (30). About all the pixels of a watch field, since the result of an 
operation is "0", a motion distinction circuit (30) judges that a motion does not exist 
all over a watch field, the case where a motion distinction circuit judges "with no 
migration body" — the following minute time amount deltat — the difference of 
brightness — it calculates. 

[0015] As shown in D from drawing 3 B, the case where the tip of a robot's (12)'s arm 
(31) invades all over a watch field (14) is considered. If the difference of a luminance 
signal is calculated in a difference circuit (26), since change has arisen in the 
luminance signal of the applicable part in a watch field (14) about drawing 3 B and C, 
the difference does not serve as zero, this difference — if the result of an operation is 
absolute-value-ized and carries out binarization in an absolute value-ized circuit (27), 
as shown in drawing 3 B-1, only the part into which the arm invaded will be set to "1" \ 
and the other part will be set to "0." Since a motion will exist all over a watch field by 
this, a motion distinction circuit (30) is judged to be "those with a migration body." 
When a motion decision circuit (30) judges it as "those with a migration body", mask 
processing of the data value of the inside by which binarization was carried out is 
carried out in a mask circuit (32) in the part (33) surrounded by "1", and it is 
transmitted to a center-of-gravity arithmetic circuit (34). A center-of-gravity 
arithmetic circuit (34) calculates the center of gravity of a mask part, and memorizes 
the coordinate in center-of-gravity memory (36). As shown in drawing 3 C and D, 
when a robot's (12)'s arm (31) next invades further all over a watch field (14), it asks 
for the difference of a luminance signal similarly, and by the same processing as the 
above, as shown in drawing 3 C-1 , the data value by which binarization was carried out 
carries out mask processing of the part (33) surrounded by "1", and the barycentric 



coordinates are calculated. The called-for barycentric coordinates and the 
barycentric coordinates memorized by center-of^gravity memory (36) are transmitted 
to a center-of-gravity difference circuit (38), and the difference between barycentric 
coordinates is called for. this difference — the difference of the coordinate searched 
for by the operation serves as a motion vector (50) of the center of gravity of a 
migration body, and ( drawing 3 D~1 reference). The called-for motion vector is 
transmitted to a direction distinction circuit (40). it is the direction (following the 
"direction of appearance") where a direction distinction circuit (40) takes out only the 
direction component in a motion vector, and the migration direction of a migration 
body separates from a robot (12) — or it distinguishes whether it is the direction 
(following the "close direction") which faces to a robot (12). 

[0016] When a direction distinction circuit (40) judges the migration direction of a 
migration body to be the direction (the direction of appearance) which separates from 
a robot (12) Usually, since [ by which a migration body trespasses upon a watch field 
(14) from a robot side as shown in D from drawing 3 B ] it is thought that it moves, 
that is, is a robot's (12)'s overrun, A control circuit (42) intercepts a robot's (12)'s 
power circuit (43), and stops supply of a power source to a robot (12). On the other 
hand, when a direction distinction circuit (40) judges the migration direction of a 
migration body to be the direction (the close direction) which faces to a robot (12), 
migration bodies, such as a man or other robots, are the motions which trespass upon 
a watch field (14), and since it is not a robot's (12)'s overrun, as for a control circuit 
(42), a signal is transmitted to an alarm means (44). An alarm means (44) emits an 
alarm with the instruction from a control circuit (42) on the migration body which 
invades with light or a sound. Thereby, the collision with a robot and a migration body 
can be prevented. 

[0017] When it is judged as "those with a migration body" irrespective of the invasion 
direction of a migration body in a motion distinction circuit, the current supply to a 
robot may be suspended or you may make it emit an alarm in the above-mentioned 
example, although the invasion to the watch field of a migration body from the motion 
vector in a watch field and its invasion direction were detected. 

[0018] You may ask for a motion vector using a representation point matching method, 
the gradient method (gradient method), etc. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the circuit diagram of the robot supervisory equipment of this 
invention. 



t 



[Drawing 2] It is the explanatory view showing the condition of supervising the robot 
using an image pick-up means. 

[Drawing 3] It is the explanatory view showing the difference of the image of an image 

pick-up means, and the brightness in a watch field. 

[Description of Notations] 

(10) Supervisory equipment 

(12) Robot 

(14) Watch field 

(16) Image pick-up means 

(18) Analysis means 
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